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Abstract 

Aim: In this study, we aimed to adapt and determine the psychometric properties of the Turkish version of the Post-Intensive Care Syndrome Questionnaire. 
Material and Methods: This methodological study was conducted with 120 patients discharged from different intensive care units in Turkey between May 2020 
and June 2021. Data were collected using a Socio-Demographic Form, Post-Intensive Care Syndrome Questionnaire, the General Health Poll, and the Pittsburgh 
Sleep Quality Index. Language and content validity were studied to adapt the scale. The construct validity was analyzed using confirmatory factor analysis. 
Reliability was evaluated with Cronbach's alpha coefficient, item-total score correlation, and similar measurement tools. 

Results: As a result of confirmatory factor analysis, the 10-item and 3-factor structure of the scale was determined. The factor loads of the items were 
between 0.60 and 0.90. Cronbach's alpha was determined as 0.94. There was a strong positive correlation between scales. 

Discussion: The Turkish version of the Post-Intensive Care Syndrome Questionnaire is a valid and reliable instrument for the Turkish population. 
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Introduction 

Intensive care units (ICU) reduce mortality rates by increasing 
the chance of survival of patients with technological 
developments and advances in medicine [1]. This especially 
contributes to the controlled recovery of patients suffering 
from uncontrolled infections and sepsis, catabolic state, major 
surgical interventions, traumas, multiple organ failures, SIRS, 
and prolonged mechanical ventilation [2]. Therefore, it often 
provides a sense of security and control for patients and their 
families. Although ICU has an important place in the survival of 
patients, it has been determined that patients who experience 
intensive care have memory, attention, emotion and insomnia 
problems [3-5]. These problems have been termed “post- 
intensive care syndrome” (PICS) by the Society of Critical Care 
Medicine [6]. PICS is a cognitive, physical, and mental disorder 
that occurs during ICU stay or after discharge from an ICU and 
involves the long-term prognosis of ICU patients and its effects 
on the patient's family [7]. These problems create serious 
obstacles both to discharge from the hospital to the home 
environment and to adapting to daily life after returning home 
[3]. 

Prolonged stay of patients in the ICU causes an increased risk 
of long-term physical, cognitive and mental complications. 
Risk factors for PICS include advanced age, delirium, acute 
brain dysfunction, hypoxia, hypotension, glucose dysregulation, 
sepsis, sedation, mechanical ventilation, and premorbid mental 
and physical comorbidity [8,9]. 

Post-ICU patients may suffer from physical problems like 
weakness, dysphagia, wasting syndrome, 
sexual dysfunction, etc., as well as mental health problems like 
depression, anxiety, panic disorder, or posttraumatic stress 
disorder (PTSD) [4,8]. Torres et al. [9] identified depression, 
anxiety, PTSD, weakness, and movement disorder in patients 
with intensive care experience. In the study of Chung et al. [10], 
it was stated that the patients who remained on mechanical 
ventilation for a long time in the ICU experienced nightmares, 
panic disorder, anxiety, difficulty in breathing and the feeling/ 
fear of suffocation. In addition, Colbenson et al. [1] reported that 
most of the patients staying in the ICU experienced cognitive 
problems for a long period of their lives. They stated that the 
physical effects of the intensive care experience also impair the 
patient's quality of life and as a result, they may have difficulty 
in continuing their daily lives. They also reported that frequent 
re-admission to the ICU may lead to trauma in patients and 
their families. 

Many assessment tools are used to evaluate the cognitive, 
physical, and mental difficulties experienced by patients with 
intensive care experience in Turkey [11-13]. However, there is no 
scale where these difficulties of patients were evaluated with 
a single scale. Therefore, the aim of this study is to evaluate 
the psychometric analysis of the Post-Intensive Care Syndrome 
Questionnaire. 


dyspnea, pain, 


Material and Methods 
Participants and Setting 
This study was conducted between May 2020 and June 2021 
with 120 patients (considering the number of scale items for 
the 18-item scale) in the ICU of an education and research 


hospital in Turkey. The literature recommends that the sample 
size should be 5-10 people for each scale item in validity and 
reliability studies [14,15]. The criteria for inclusion were age 
over 18 years, experience of staying in the ICU for at least 
2 nights, at least 1 month and not more than 1 year after 
leaving the ICU, and voluntary participation in the study. The 
list of patients discharged from the ICU was accessed through 
the hospital's automation system. Data were collected via 
telephone. Before the study began, patients were contacted by 
phone to inform them about the purpose and process of the 
study. 

Instruments 

Socio-demographic Form: It was used to collect socio- 
demographic information of the patients. 

Post-Intensive Care Syndrome Questionnaire (PICSQ): It was 
developed by Jeong and Kang [16]. The 4-point Likert-type 
(O=Never, 2=Most often, 3=Always) 
consists of 18 items and three sub-dimensions. These are 
cognitive (1st, 2nd, 3rd, 4th, 5th, 6th items), physical (7th, 8th, 
9th, 10th, 11th, 12th items), and mental (13th, 14th, 15th, 16th 
17th, 18th items) sub-dimensions. The total score of PICSQ is 
between 0 and 54, or the mean is between 0 and 3. High scores 
show that the level of PICS is high. The Cronbach's alpha for the 
PICSQ was 0.94 and its sub-dimensions were between 0.87 and 
0.95 in this study. 

General Health Poll (GHP): It was developed by Goldberg [17] 
and adapted in Turkish by Kilig [18]. It consists of 20 items ona 
4-point Likert-type scale. The minimum and maximum scores of 
the scale are O and 36. High scores indicate that the incidence of 
mental problems (anxiety and depression) increases. Cronbach's 
alpha for the GHP was 0.75 for this study. 

Pittsburgh Sleep Quality Index (PSQI): It was developed by 
Buysse et al. [19] and tested for the Turkish language by Agargtin 
et al. [20] to evaluate the sleep quality in the last month. The 
index includes 24 questions, nineteen of these are self-report 
questions and answered by the patient, five questions are 


1=Sometimes, scale 


answered by a spouse or roommate. These five questions are 
not included in the scoring and are therefore used for clinical 
information only. The last of the self-report questions (question 
19) is about the availability of a roommate or spouse and is not 
used in scoring. The total score is between 0-21. A total score 
higher than five indicates poor sleep quality. In this study, the 
Cronbach's alpha for the PSQI was 0.69. 

Ethical Consideration 

Permission was obtained from the corresponding author of the 
original PICSQ [16] and from the University Ethics Committee. 
The study was conducted according to the Declaration of 
Helsinki and consent was obtained from the patients who 
volunteered to participate in the study. 

Statistical Analysis 

For the analysis of the data, SPSS 21.0 and AMOS 22.0 statistical 
programs were used. Number, percentage, mean and standard 
deviation were calculated for descriptive statistics. Language 
and content validity were studied during the adaptation 
process. Confirmatory factor analysis (CFA) was used within 
the scope of the validity, and item-total correlation, Cronbach's 
alpha coefficient, and equivalent form analyses were used 
within the scope of the reliability of the study. CFA evaluates 
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whether a previously defined and constrained construct has for the other items in the factor to which the items belonged. 
been validated as a model. CFA is one of the structural equation = For the stated reasons, the items with high correlation with the 
models, and in structural equation models, the model fit must items in other factors were gradually removed from the scale 
be ensured first. In the evaluation of model fit, "Chi-square (primarily with the highest covariance correlation value), and the 
statistics to the degree of freedom ratio" (X2/df), "statistical | model fit indices were tried to be improved. After the remaining 
significance of individual parameter estimates" (t value), 10 items and three covariance connections in the scale (item3- 
"standardized root-mean-square residual" (SRMR), “goodness- —_item5, item9-item11, item16-item18), it was determined that 


SP TieInde (GF), MOnsnonMed Tt Inde ANG, “competative Table 1. Model fit indices obtained in the confirmatory factor 


fit index” (CFI) and “root mean square error of approximation analysis (CFA) of the PICSQ 


(RMSEA)” were used. Structural Equation Modelling was also 
applied. Pearson correlation analysis was applied to determine The first CFA The last CFA’ 

the relationship between the scales. For statistical significance, Model Fit Indices -=—=—S—S«S items —~=S«é«SC‘C items «St*«C eect 
p<0.05 was accepted. Tan Soe 


Fit® 


3 sub-dimensions 3 sub-dimensions 


X2/df 4.258 2.196 <3 
Results SRMR 0.093 0.039 <0.05 
Of the patients, 60.8% (n=73) were females, their mean age GFI 0.655 0.906 >0.95 
was 53.68+13.68 years, 66.7% (n=80) were hospitalized in the NNFI 0.772 0.950 20.95 
surgical ICU, 57.5% (n=69) were admitted electively, and 23.3% CFI 0.803 0.968 >0.95 
(n=28) received mechanical ventilation. The mean length of RMSEA 0.165 0.100 <0.08 
stay in the ICU was 3.33+2.33 days, and the mean time after Factor load 0.60 / 0.93 0.69 / 0.93 
discharge was 5.64+3.47 months, and 13.3% (n=16) were re- ee between 0.74/0.78/0.86 0.79/0.79/0.79 
hospitalised. Explained total variance * 77.85% 84.52% 
Validity 


Abbreviations: * Tested by exploratory factor analysis, ° Cokluk, Sekercioglu, and 
The findings obtained in the CFA performed with the 18 items Buylk6zturk, (2010), " With covariance connections 


and 3-sub-dimensional structure of the PICSQ are given in 
Table 1. According to the results of the CFA, it was determined Table 2. CFA Results of PICSQ 
that the item factor loads were quite high, but the values of 


Cronbach's 
the model fit indices were not in the appropriate ranges. When Item and Sewer ee ree Alpha 
: : . ‘ Sub-dimension 
the suggested covariance connections were examined, it was (0.94) 
determined that the items had a high correlation with the items Cognitive (%36.23) 
in the other factors, despite the high factor loading in the factor rome 0.89 O87 018 0.50 0.78 
they belonged to. This indicates that the scale has a low level tera O8e 1372 2081) .0.50)./024: 078 
0.95 
of discrimination and, accordingly, the model fit indices cannot fem? DB AOE O86 S01 203i 077 
reach a sufficient level. On the other hand, it was determined sem O24 1099 089 029022 082 
5 ee ri , a, Ne : Physical (%24.77. 
that there was no significant improvement in the fit indices, eng) 
. a tem 7 0.93 0.52 0.67 0.26 0.80 
although the suggested covariance connections were made = 
tem 9 0.69 8.44" 0.21 0.90 015 0.64 0.89 
tem 11 0.79 10.317 0.22 0.84 037 0.74 
Fen on Mental (%23.51) 
” a | — orion ~~ 
/ Ly ~~ a tem 14 0.86 0.43 0.38 0.60 0.76 
~-_ ——— ~ a ™~, 
(a ~ \ " 
x ( a ) ——> ptt i s~ / < item 16 0.72 8.08 023 0.19 089 0.66 0.87 
\ > = Je y tem 18 079 911° 033 028 082 074 
f an ea ~ — is 
a/ bl } Pm ” \ Abbreviations: r: Item Total Correlation, ‘p<0.01 
(«) ~ ys 
e+ intant : A 
a } \ Table 3. Equivalent Form Correlation Results 
\ 
(aa / \ 
" aw -— poe ——_ A Al A2 AZ B Cc XxX sD Ss. K 
= ——— ~ 
fw f n 
12)\—r ae 
‘A, a PN 2 a 1 067° 090° 087° 0.38 042° 1486 5.65 1.83 3.47 
= —, - ~ - \ 
iv a \ i 
a \ ane 1 051° O55 027° 025 512 244 267 7.08 
— —_—_—_—_—_—_—_—— | / 
” Pt 
/ Physical 1 066 025° O31° 532 220 091 032 
co l/ (A2) 
/ 
od -—— Baronet Uf Mental 6 : 
Sg ~~ tt oy (A3) 1 0.38 0.42 441 Unkss ss) 2 
~. 
‘ n wots } 
At) —— | eel pt J aa 1 048 «413.21 497 055 0.96 
a -/ . toe NS - (8) 


Lr a ~ — 
- PSQI 
g © 1 11.95 171 1.27 250 
/ 


Abbreviations: PICSQ: Post-Intensive Care Syndrome Questionnaire, GHP: General Health 
Poll, PSQI: Pittsburgh Sleep Quality Index, X: Mean, SD: Standard Deviation, S.: Skewness 


Figure 1. CFA diagram of PICSQ K. Kurtosis, ‘p<0.01 
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the model fit indices reached good and very good levels, and 
the factor loads remained within the appropriate ranges (Table 
1). 

The results of the validity analysis, consisting of factor load 
and t values, are shown in Table 2. As a result of the CFA, it was 
determined that the factor loads of the remaining 10 items in 
the scale were higher than 0.40, and the t values of the items 
were significant (p<0.01). It was also determined that the total 
scale had 84.52% of the total variance. According to the results 
obtained, it was determined that the PICSQ is a valid scale with 
10 items and 3 sub-dimensional structures. The verified model's 
CFA diagram is shown in Figure 1. 

Reliability 

Cronbach's alpha was evaluated to determine the internal 
consistency of the scale. Cronbach's alpha was 0.94 for the 
scale, and it was between 0.87 and 0.95 for sub-dimensions. 
It was determined that the item-total correlation for all items 
ranged from 0.64 to 0.82 (Table 2). 

The minimum and maximum scores of the PICSQ are O and 
30, and the mean score was determined as 14.86+5.65. The 
minimum and maximum scores of the cognitive, physical, 
and mental sub-dimensions of the PICSQ are 0-12, 0-9, and 
0-9, respectively. The mean scores of the cognitive, physical, 
and mental sub-dimensions of the PICSQ were 5.12+2.44, 
5.32+2.20, and 4.41+1.78, respectively. The minimum and 
maximum scores of the GHP are O and 36, and the mean score 
was 13.21+4.97. The minimum and maximum scores of the 
PSQI are O and 21, and the mean score was 11.95+1.71 (Table 
3). 

For the scale’s equivalent form reliability analysis, the correlation 
results between PICSQ, GHP, and PSQI are given in Table 3. 
Correlation values of the relationship between PICSQ total 
scores and GHP were determined as 0.38; correlation values 
of the relationships between the sub-dimensions of PICSQ and 
GHP were determined between 0.27 and 0.38 (p<0.01). When 
the correlations between PICSQ and PSQI were examined, 
the correlation between the total scores was 0.42, and the 
correlations of the sub-dimensions were between 0.25 and 0.42 
(p<0.01). These correlation results show the consistency of the 
PICSQ. 


Discussion 

The Turkish validity and reliability of the PICSQ were evaluated 
in this study since there is no comprehensive tool that can be 
used in Turkey to evaluate the PICS in patients with intensive 
care experience. This study was conducted with 120 patients. 
The literature recommends that the sample size should not 
be less than 5-10 times the number of scale items in order to 
perform factor analysis in scale studies [14,15]. For this reason, 
at least 5 participant rules were provided for each item of the 
18 items used in the assessment. 

Validity 

In this study, the construct validity of the PICSQ was determined 
by factor loads, and it was determined that although the factor 
loads were quite high, the values of the model fit indices were 
not in the appropriate ranges. For this reason, the proposed 
covariance connections were examined and 8 items (1, 2, 8, 
10, 12, 13, 15, 17) that had a high correlation with the items 


in the other factors, despite the high factor load in the factor 
they belonged to, were gradually removed from the scale. As a 
result of CFA, the model fit indexes of the remaining 10 items 
in the scale were determined as x2/df=0.196; SRMR=0.039; 
GFl=0.906; NFI=0.950; CFIl=0.968; and RMSEA=0.100 (Table 1). 
The results of this study meet the perfect fit criteria specified 
in the model fit index results in the literature [21,22]. In the 
study, it was determined that the scale covered 84.52% of the 
total variance. In addition, in this study, the factor loads of the 
items were between 0.60 and 0.90, and a 3 sub-dimensional 
structure was determined (Table 2). This result meets the 
recommendation in the literature that factor loads should be 
greater than 0.40 and explain at least 30% of the variance 
[14]. In addition, it was determined that the 3 sub-dimensional 
structure of the scale was similar to the original scale [16]. 
Reliability 

Cronbach's alpha coefficient used to evaluate the internal 
consistency of this scale was calculated as 0.94 (Table 2). 
In scale validity and reliability studies, it is recommended 
to calculate Cronbach's alpha coefficient to determine the 
reliability of the Likert-type scale [23]. Cronbach's alpha 
coefficient of the original scale was reported as 0.93 [16]. It 
is stated that the scale is not reliable if Cronbach's alpha is 
0.00<a<0.40, it is low reliable if 0.40<a<0.60, it is reliable if 
0.60<a<0.80, and it is quite reliable if 0.80<a<1.00 [14]. Since 
the Cronbach alpha value was determined to be higher than 
0.70 in the Turkish version of the PICSQ, it can be said that the 
scale is a very reliable instrument. 

Another test used to assess internal consistency is item-total 
score correlation. It is stated that the total score correlation of 
an item should be at least 0.30 statistically [24]. In this study, 
each of the item-total correlations of the scale was determined 
above the recommended minimum level (0.64-0.82). Therefore, 
it can be said that the internal consistency of the scale and all 
its items is high (Table 2). 

It is recommended to either re-test the scale or use equivalent 
tests to determine the invariance of the scale over time [14,24]. 
In this study, a re-test was not performed because of the PICS 
instability in the patients after discharge from the ICU. This 
problem was solved using similar scales. In this study, GHP and 
PSQI were used as equivalent tests to determine the invariance 
of the scale over time. In the correlation analyses performed 
between the scales, the correlation values between PICSQ 
total and GHP and PSQI were found to be 0.38 and 0.42 and 
statistically significant (p<0.01) (Table 3). When correlation 
values are evaluated as O-0.2=very weak, 0.2-0.4=weak, 
0.4-0.6=moderate, 0.6-0.8=strong, and 0.8-1.0=very strong 
[23], it can be said that there is a strong and significant positive 
correlation between the scales. 

Limitations 

This study was conducted via telephone, but not face-to-face. 
Therefore, a formal environment could not be created. This 
study was conducted with patients discharged from different 
ICUs of a single hospital. In addition, other cognitive, physical, 
and mental factors affecting patients could not be controlled. 
Conclusions 

When the validity and reliability analyses were evaluated 
together, it was determined that the Turkish version of the 
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PICSQ was a reliable and valid scale with 10 items and a 
3-dimensional structure. PICSQ can be accepted as a valid 
and reliable tool to evaluate PICS in adults with intensive care 
experience in Turkey. It can be recommended to apply the scale 
to different and wider populations in Turkey. 
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